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ABSTRACT 

This study assessed the extent to which students 
commit various types of errors when completing Holland's 
Self-Directed Search (SDS) entirely on their own. Nearly all students 
made some type of error and approximately half of the students made 
errors that affected their final 3-letter summary codes. Almost 
one-fifth of the students made errors resulting in an incorrect 
high- point code. Whether or not students made errors that affected 
their summary codes was unrelated to the extent to which they were 
interested in knowing more about occupations or academic majors they 
might like. Neither were such errors related to whether students felt 
the occupations suggested by their summary codes seemed reasonable. 
Only slightly over one-third of the students actually did feel that 
the occupations seemed reasonable. (Author) 
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THE ACCURACY OF SELF-ADMINISTRATION AND SCORING 



ON HOLLAND'S SELF-DIRECTED SEARCH 
Abstract 

This study assesed the extent to which Ss commit various types of errors 
when completing Holland's Self-Directed Search (SDS) entirely on their own. 
Nearly all Ss made some type of error and approximately half of the Sis made 
errors which affected their final three-letter summary codes. Almost one- 
fifth of the Ss made errors resulting in an incorrect high-point code. Whether 
or not Ss made errors that affected their summary codes was unrelated to the 
extent to which they were interested in knowing more about occupations or ac- 
ademic majors they might like. Neither were such errors related to whether 
Ss felt the occupations suggested by their summary codes seemed reasonable. 

Only slightly over one-third the Ss actually did feel that the occupations 
seemed reasonable. 
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THE ACCURACY OF SELF-ADMINISTRATION AND SCORING 



ON HOLLAND'S SELF-DIRECTED SEARCH (SDS) 

There has been a major need in vocational psychology and counseling For 
a short, self-administering and interpreting instrument that efficiently sum- 
marizes an individual's interests and abilities and, at the same time, links 
them to relevant occupations. Most vocational interest inventories must be 
machine scored at a central location. In addition, if the results are being 
used for guidance purposes they must be interpreted to the counselee by a 
trained counselor. Ideally, an instrument which the individual could score and 
interpret himself would eliminate the time lag involved in machine scoring and 
even obviate much of the need for face-to-face counseling. Thus, the instru- 
ment would be highly economical in 'terms of time, effort and money. Such an 
instrument, part of a larger vocational -guidance system, has recently been 
developed (Holland, 1971a) and preliminary research suggests that simply com- 
pleting (including sel f- i nterpretat i on) it has a desirable effect on the voca- 
tional development of the subject (Zener £• Schnuelle, 1972). 

The Self-Directed Search: A Guide for Educational and Vocational Planning 

(SDS; Holland, 1971a, 1971b) is a self-administering, self-scoring and self- 
interpreting instrument based on Holland's (1966) theory of vocational choice. 

The theory asserts that the choice of a vocation is a reflection of one's person- 
ality, and that essentially there are six vocat iona 1 1 y-rel evant personality types: 
realistic, investigative, artistic, social, enterprising, conventional. Each 
type has a corresponding (same-named) occupational environment with which it is 
compatible. Each of the occupational environments, in turn, consists of a number 
of occupations which reflect that environment. Thus, the SDS enables people to 
assess their personality types (resemblence to each of the six theoretical types) 
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and determine the resulting occupations for which they would be suited (Holland, 
Viernstein, Kuo, Karweit 6- Blum, 1970) 

A critical question about an instrument such as the SDS focuses on 
whether individuals who administer it to themselves proceed to score it 
accurately. Inability to do so would seriously attenuate the instrument's 
reliability and, of course, validity. Thus, the major purpose of the present 
study was to examine the accuracy of self-administration and scoring on the 
SDS. Research indicates that motivated subjects make fewer errors in taking 
tests than do non-motivated Ss (Anastasi, 1968). Relatedly, the second pur- 
pose of the study was to gather evidence on whether the accuracy of self- 
administration and scoring on the SDS was affected by subjects' degree of 
interest in learning more about what occupations and academic majors they 
might like. A final objective was to determine if the extent to which Ss 
agree with their results on the SDS was related to accuracy of self-adminis- 
tration and scoring. 

Method 

Subjects and Administration 

As part of a summer orientation program, 221 incoming freshmen at the 
University of Maryland, College Park, completed the SDS in one group. After 
briefly describing the SDS, proctors distributed it to the J5s and left the 
testing room, returning after giving all Ss time enough (60 minutes) to com- 
plete it. Three items were added to the Assessment Booklet (described below) 
of the SDS. On the front cover, Ss were asked to indicate the extent to 
which they were interested in knowing more about what occupations they might 
like and what academic major they might like. Ratings for these items were 
on a four-point Likert scale (1 = not interested, k = very interested). After 
completing the SDS £s were asked to rate the extent to which they agreed that 
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their final summary codes were reasonable (1 strongly agree, 5 = strongly 
d i sagree) . 

The SDS: Self-Administration and Scoring 

The SDS consists of two booklets. S^ fills out the Assessment Booklet which 
yields a three-letter occupational code indicating the three personality types 
to which he is most similar. He then uses this code to search for suitable occu- 
pations in the Occupations Finder , a booklet that contains occupational titles 
which subsume about 95 percent of the labor force. Since the present study 
focuses on errors made in completing the Assessment Booklet, the paragraph below 
will present the procedures 5>s must follow in completing that booklet. 

^s begin in the Assessment Booklet by listing the occupations about which 
they have daydreamed and then finding the three-letter code for each occupation. 
Following this they indicate whether they like or dislike 11 acti vi ti es reflecting 
each of the six occupational environments (thus, 66 activities in sum). Ss then 
rate their competencies in the same manner, following which they indicate whether 
they woul d 1 i ke or d i s 1 i ke 14 occupat i ons reflecting each of the six env i ronments . 
The next step consists of rating on a seven-point scale self-estimates of abilities 
in 12 general areas (e.g., artistic ability, sales ability), two each reflecting 
each of the six occupational environments in Holland's scheme. From this 
derives the three-letter codes for the two sets of six general areas. The final 
steps in completing the Assessment Booklet consist of summing and plotting on 
graphs scores for each of the first three steps indicated above (competencies, 
occupations, abilities), and determining the three-letter codes for each step. 

S,s transfer to another table the number of times each of the six possible letters 
appeared first, second and third in the four summary codes for act ivi ties , com - 
petenci es , occupati ons and self estimates of abilities . The last step in complet- 



ing the Booklet is to (a) multiply by three the number of times each letter (e.g., 
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R for Realistic, I for Investigative) appeared in first place in the four sum- 
mary codes, (b) multiply by two the number of times each letter appeared in 
second place, and (c) multiply by one the number of times each letter appeared 
in third place. The end result of these operations is a score for each of the 
six letters (R I A S E C). The three highest scores compose J5's final summary 
code upon which he bases his search for suitable occupations in the Occupations 
Fi nder . 

Analysis of Errors 

The study of the accuracy of Ss 1 self-administration and scoring was made 
by three undergraduate research assistants. One assistant first re-scored all 
of the Assessment Booklets (219). Then, to assure that errors were not commit- 
ted in the re-scoring itself, the two other assistants again re-scored approx- 
imately one-half the booklets each. Thus, all booklets were re-scored by two 
assistants. In cases where two assistants' re-scoring and noting of errors 
by did not coincide (about 20 percent of the time), the two assistants dis- 
cussed their lack of agreement until agreement was reached. 

Analysis was made of six types of errors. Add i t ion errors indicated those 
made in summing his "yes" or "like" responses to the items reflecting each 
of the six occupational environments for the Activities, Competencies and 
Occupations sections of the Booklet. Rating errors reflect mistakes .S made 
in completing the graphs on Self-estimates of Ability, Activities, Competencies 
and Occupations and determining his three-letter code from each graph. Rating 
errors were cumulative in the sense that if £ made an addition error he would 
necessarily produce an incorrect graph. Summary-table errors indicate errors 
in counting the number of times each letter (i.e., R I A S E C) appeared first, 
second or third in the four summary codes for Self-Estimates, Activities, Com- 
petencies and Occupations. Summa ry-tabl e errors also reflect mistakes in 
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multiplying and summing that are involved in this last step from which S/s 
final three letter summary code is derived. To some extent summary-table er- 
rors are also cumulative. 

The re-scoring of each S/ s Assessment Booklet allowed the judges to deter- 
mine whether or not made three additional types of errors: (a) whether scor- 

ing errors produced incorrect final summary codes (one or more of the three 
letters excluded or an incorrect ordering of the three letters in the code); 

(b) whether scoring errors produced a code in which one or more of the letters 
that should have been included was not (and thus an inappropriate letter was 
included); (c) whether scoring errors produced an incorrect high-point code 
(the personality type which ^ most resembles). It should be noted that these 
last three types of errors are by far the most serious, since they directly 
affect the occupations for which jS will find himself suitable when he searches 
the Occupations Finder . 

Results 

Table 1 presents the number and percentage of Ss who made the various types 
of errors. It can be seen that nearly all Ss made some type of error and that 
most of these were made in transferring scores to graphs, determining summary 
codes for each graph and in completing the Summary Table. In about one-half 
the cases Ss made errors that did affect their final summary code (ordering 
change or appropriate letter omitted - inappropriate one included), while about 
one-fifth the Ss made errors which produced an incorrect high-point code. 



Insert Table 1 About Here 

Subjects reported a high degree of interest in learning more about academic 
majors and occupations they might like. In response to the item asking J5s to 
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rate this variable, 135 (62%) claimed to be "very interested", 55 (25%) "some- 
what interested", 19 (9%) "slightly interested" and 9 (4%) "not interested". 

The similarly-worded item asking Ss about academic majors elicited almost 
identical responses: 133 (6l%) "very interested", 53 (24%) "somewhat inter- 
ested", 21 (10%) "slightly interested", 11 (5%) "not interested". 

Perhaps because it was placed at the end of the Assessment Booklet, only 
123 (55%) of the Ss responded to the item asking them to indicate the extent 
to which they agree that their summary-code occupations seemed reasonable to 
them. A wide distribution of responses emerged on this item: 20 (16%) "strongly 

disagree", 24 (20%) "disagree", 34 ( 28 %) "neutral", 38 (31%) "agree", 7 (6%) 
"strongly agree". 

To determine if interest in learning about suitable academic majors and/or 
occupations was associated with errors on the SDS, and if errors, in turn, 
were related to Ss 1 agreement that his results were reasonable, correlations 
were computed between the relevant variables. Only the three indices of errors 
in the final summary code were employed in this analysis, since such errors 
are the ones which are genuinely undesirable. (Errors in addition, rating 
and completing the summary table are undesirable only inasmuch as they change 
the final summary code.) The intercorrelations are presented in Table 2. 



Insert Table 2 About Here 

Table 2 reveals that while S_s* interest in learning more about occupations 
and academic majors they might like are highly intercorrelated (r = .70), 
neither of these variables is associated with any of the three types of scor- 
ing errors. In addition, whether or not Ss make such errors is unrelated to 
the extent to which they feel their results are reasonable. 
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To assess whether interest in learning more about suitable occupations 
and suitable academic majors interacted in affecting errors in JSs 1 summary 
code a 2 x 2 analysis of variance for unequal jn 1 s was computed for each of 
the three measures of summary-code errors. Interest in learning about suit- 
able occupations, the first main effect, was dichotomized so the JSs who 
claimed to be "very i nterested" (n = 135 ) represented one level and those 
claiming to be somewhat, slightly or not interested (n = 83 ) represented the 
other level. The second main effect, interest in learning about suitable 
academic majors, was similarly divided into two levels (very interested -• 133; 
somewhat, slightly and not interested = 85 ). The ANOVA revealed that the 
interaction of interest in learning about suitable majors and occupations did 
not approach significance (p . 05 ) in affecting any of the three types of 
summary- code errors. 

Discussion 

In summary, nearly all Ss in the present study made some type of error 
in scoring their SDS results. Approximately one-half made errors which affected 
their final, three-letter summary code in some way (incorrect order, appropri- 
ate letter omitted), and almost one-fifth committed errors resulting in an 
incorrect high-point code. While the effect of such errors on the reliability 
of the SDS summary code cannot be assessed directly from these data, O'Connell 
and Sedlacek's (1971) research provides suggestive evidence. These researchers 
obtained coefficients of stability for corrected summary codes that were appre- 
ciably higher (.20 or more for four of the six scales) than coefficients re- 
ported by Baldwin (1970 for uncorrected codes, despite the fact that the 
time between the initial and later test was 7-10 months in the O'Connell- 
Sedlacek study and only three weeks in the Baldwin study. Thus, it would 
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appear that the kinds of errors revealed in the present study have a serious 
effect on the reliability of the SDS. 

It will be recalled that the SDS was administered during freshman orien- 
tation and that the administrators left the testing room (a proctor remained) 
following distribution of the instrument. Thus, the SDS was truly self- 
administered and scored. This raises the question, however, of whether the 
error rates may have been reduced by careful monitoring. While a definitive 
answer is not available, recent research does suggest an affirmative answer. 

In the O'Connell and Sedlacek (1971) study, where careful monitoring was con- 
ducted, only 71 percent of the Ss, as compared to 92 percent in the present 
study, made some type of error. In addition, Zener and Schneulle (1972) 
found lower error rates than in the present study, despite the fact that they 
gave the SDS to high school students. In their investigation one monitor per 
25 j5s was available to answer questions. Taken together these studies sug- 
gest that, while the SDS is purported to be self-administering and scoring 
(Holland, 1 97 1 a » 1971b), it is important to have monitors available to help 
with scoring questions, at least when the instrument is used with large groups 
(one of its suggested uses, cf. Holland, 1971a. p6). 

It was surprising that Ss 1 expressed interest in learning more about 
suitable occupations or majors was unrelated to whether they made scoring 
errors on the SDS. It was assumed that such interest would affect Ss 1 inter- 
est in taking and carefully scoring the SDS, but it appears that this may not 
be the case. Additional research is needed to examine the effect of actual 
interest in taking the SDS on error rates. However, the present results do 
suggest that seif-reported interest in learning more about suitable occupations 
or majors is not a variable in SDS scoring accuracy. 
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The finding that errors which affect the final summary code are unrelated 
to the extent to which Ss feel their summary-code occupations are reasonable 
may seem to indicate that summary-code errors do not have consequences that 
are as negative as one might expect. Instructions in the Assessment Booklet, 
for example, ask _Ss to search the Occupations Finder for all possible arrange - 
ments of their code. Thus, an appropriate letter would have to be omitted 
from the three-letter code (and an inappropriate one included) for j> to be 
unable to identify an appropriate occupation in the Finder. Such an explana- 
tion is negated, however, by the fact that errors resulting in the omission 
of an appropriate letter from the code were unrelated to feeling that occu- 
pations suggested by the final code were reasonable. A more likely explana- 
tion of the lack of association of errors to degree of agreement-disagreement 
is that incoming freshman know relatively little about what occupations are 
compatible with their personalities. As a result of this lack of knowledge, 
occupations based on an incorrect summary code (not entirely incorrect) seem 
as reasonable to such students as do occupations derived from a completely 
accurate code. 

It seems noteworthy that only 37 percent of the Ss who responded to the 
item on reasonableness either agreed or strongly agreed that their summary- 
code occupations seemed reasonable. Conversely, 36 percent either disagreed 
or strongly disagreed and 28 percent gave a neutral response. Such a distri- 
bution indicates that the SDS (particularly when it is truly self-administered, 
scored and interpreted) may have less appeal to university students than the 
author of the instrument has suspected (Holland, 1971b, p. 175). Since the 
notion of a self-directed search does appear to have much merit (cf. Holland, 

1 97 1 a > 1971b), research is needed to determine why such a relatively large 
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proportion of students may not feel their results in the SDS are reasonable. 

A beginning study along these lines by Collins and Sedlacek (1973) revealed 
that entering college students who obtained SDS summary codes linked to occu- 
pations not requiring college tended to be dissatisfied with their results. 

At a minimum the present results imply that unaided completion of 
Holland's SDS by incoming college freshmen during orientation results in 
an appreciably higher error rate than would seem desirable. While at least 
one study (Zener & Schnuelle, 1972) supports the value of the instrument in 
vocational guidance, the reliability (and thus validity) of the SDS is atten- 
uated by scoring errors. The authors' impression is that many such errors 
could be eliminated by a clearer organization of the Assessment Booklet and 
a simplification of the self-scoring procedures. Such steps seem warranted 
in light of the many advantages Holland (1971b) had indicated are inherent 
in a self-directed search for one's educational and vocational goals. 
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Table 


1 




Number and Pet. of Ss 


Maki ng 


Different Types of 


Errors 


Type of Error 


Number 
of Ss 


' Number 

Making Errors 


Percentage 
Making Errors 


Add i t i on 


221 


42 


19 


Rati ng 


221 


206 


92 


Summary Table 


217 


199 


89 


Errors Affecting Final Code 


217 


122 


55 


One or More Appropriate Letters 


217 


104 


47 


Omitted in Code 








High-Point Code Incorrect 


217 


40 


18 



' 22 1 Ss began the SDS; 4 of these discontinued when they reached the summary 
tabl e. 



15 




Table 2 

Relationship of Errors to Interest in Knowing More About Suitable Majors and Occupation and to 



